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ABSTBACT 

This paper considers hov to respond to nev 
requirements for adequate disclosure of the schools' performance to 
the public. It proposes the use of three powerful constructs — quality 
control, quality assurance, anc* an independent educational 
accomplishment audit (lEAA). Tht* essential elements of quality 
control are agreeing on and specifying desired outcomes of 
instruction (the standards), translating the desired outcomes into 
measurable objectives or verifiable proxy experiences (the design) , 
formally comparing what is actually happening vith what vas intended 
(the evaluation for conformance to design) , deciding nhat is required 
to be done to achieve conformance to design (the control plan) , and 
doing vhat is required vhen such action is feasible (the control 
action)* The agenda of quality assurance is the organization's 
quality control program. An lEAA is an audi! of program 
accomplishment, in output terms, done by an outside agency* The audit 
vould consider the quality control in a program and the program's 
cost-effectiveness. The intention of an audit must be vieved as 
positive and necessary if organizations are to learn and change for 
the better. Ideally, the net effect of an educational audit will be 
an emergence of an increasing number of success strategies* 
(Author/IBT) 
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EDUCATIONAL AUDITING AND QUALITY ASSURANCE 

In spite of continued assaults upon it, the "people's 
right to know" remains a fundamental tenet of all democratic 
societies. In education, we have the opportunity to demon- 
strate belief in this principle, not with preachments but with 
action; not by timidity but by firm resolve. We ci.u seize the 
Initiative to respond to new requirements for adequate dis- 
closure of the consequences of our stewardship. How to do 
this, is the subject of this paper. We propose the utilization 
of three powerful constructs: quality control , quality assur- 
ance , and an in-iei^endent educational acco mplishment audit 
XIIaa).^ To be considered are: what an educational audit is 
and is not; th relationship between quality control and quality 
assurance; pit alls to avoid; functions of the audit; the 
spirit of the audit; and necessary safeguards. 



Basic Differences Between Quality Control a nd Quality 
Assurance^ 

Quality control is internal; Evaluation is formative and 
interventive. Quality assurance is external; Evaluation is 
summative and interventive. The principal tool of quality con- 
trol is a dynamic information management system; with quality 
assurance, the principal tool is the external audit with its 
resulting report. With both processes, the operating adjective 
is responsive . Stated in simplest terms, quality control means 
for iatlVr"ivaluation Plus timelv remediation ; quality assur- 
ancp means summative evaluation for the p urpose of feedback and 
redesign . 



The Audit 

If the educational accomplishjnent audit is to have face 
validity, it must be externally acministered. The precedent 
for external' audits is firmly established in government at all 
levels, in all social institutions, and private business — 
indeed, in every school district in the country. True, we are 
talking about a fiscal audit. But why should we audit the way 
in which money is spent and not audit the results achieved 
with our resources? 

Actually, program auditing has been going on a long time. 
Granddaddy of such audits is the Genera] Accounting Office 
(GAO), an arm of Congress. Every federal law is subject to 
the scrutiny of GAO. So is the President. The audit has 
three foci: 

TT^EaF^s first described in Ever^ | Winner, by Leon M. 
Lessinger (Science Research Associates, 1970). 
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1- Compliance with the law; 

2- Efficiency , Was the money allocated 
utilized In the most efficient manner? 

3^ Effectiveness , Were th^ objectives 
achieved? 

To put matters In true perspective, there have been educa- 
tional audits for decades: various accrediting associations 
have long performed useful audits In higher education and pre- 
colleglate Institutions. Unfortunately, these audits have two 
glaring defects: (1) Although most educational auditing (or 
study) teams come from outside the system, the audit remains 
by and large. Internal; (2) auditing standards are almost ex- 
clusively based on Input-Indicators. It Is noteworthy that 
many accrediting groups are currently working diligently to 
correct this latter condition. But, much more needs to be done. 

Interestingly, there may be a significant breakthrough for 
the lEAA. Its Impetus comes not from the educational establish- 
ment, but from the Greater Dallas Chamber of Commerce and the 
U.S. District Court In Texas. In an "Opinion and Order" 
(Eddie Mitchell Tasby and Philip Wayne Tasby et . al.vs. Dr. 
Nolan Estes, General Superintendent, Dallas? Independent School 
District et. al.). Judge W. M. Taylor directs the Dallas ISD to 
"secure the service of an Independent professional firm" to 
conduct an Independent audit and report; to the court annually. 
A related development Is the court's endorsement of the Dallas 
Chamber's concept of accountability. This, in Itself, may 
represent a landmark development in Judicial decisions. 

Los Angeles County, under the leadership of Dr. Richard 
Clowes, Superintendent, has installed the most comprehensive 
educational auditing program in the country. 2 in Montgomery 
County, Maryland, Superintendent Charles M. Bernardo established 
a Department of Quality Assurance, designed to create and imple- 
ment an lEAA, and all it implies. 



Defining Quality 

In the first paper of this series ("Quality Control: the 
Missing Link in Educational Management"), quality was defined 
as fitness for use as Judged by the user . This definition 
applies equally to control and assurance. 



2/ For more information, contact Dr. Gordon Footman, Director, 
Division of Program Evaluation, Research and Pupil Services, 
Los Angeles Co. Education Center, 9300 E. Imperial Hwy., 
Downey, Ca. 90242. 
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Basic Elements of Quality Control 

Since quality assurance Is Integrally related to quality 
control. It Is useful to Identify the essential elements of 
quality control: 

(1) agreeing upon and specifying desired outcomes 
of Instruction (the standards ) ; 

(2) translating the desired outcomes Into measurable 
objectives or verifiable proxy experiences 

(the design ) ; 

(3) formally comparing what Is actually happening 
with what was Intended (the evaluation for 
conformance to design ) : 

(4) deciding what Is required to be done to achle • 
conformance to design (the control plan ); 

(5) doing what Is required when such action is 
feasible (the control action )* 3 

Thus, the agenda of quality assurance is the organization's 
quality-control program. A synergistic relationship exists 
between quality control and quality assurance. It follow: that 
lEAA Ideally should be implemented following the establishment 
of a program of quality control. If the audit is performed 
before a quality control program has been Installed, the level 
of threat is likely to be very high. Conversely, in organiza- 
tions with sound quality-control programs, the audit will be 
relatively painless. For those organizations that do not have 
formal quality-control programs, the use of an orsaiiizational 
audit is si^ggested.^ This approach allows an analysis of the 
system, specifically in relation to quality control. 



Quality and the Zero-Reject Ideal 

When deslgr.ing anything, few people set out to create the 
ordinary. Our designs are likely to represent ideal si:ates. 
The polar star of educational planning is an ideal or ultimate 
standard. Achievement of articulated objectives is the ideal 
standard in education. If every student achieves every ob- 
jective, then the ideal state is reached. 

3/ Leon M. Lessinger, "Quality Control and Quality Assurance 
in Education," Journal of Education Finance > Spring, 1976, 
p. 504. 

4/ One Instrument to perform such an audit is Independent 
Educational Management Audit : A System Approach , by James 
E. Conner, Leon M. Lessinger, Roger A. Kaufman, and Richard 
L. McVity (Washington, D.C., College/University Press, 
21i*l Wyoming Ave., N.W., 1973)* 
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It's possible to define a zero-reject Ideal In symbolic 
terms for use In a quality control program. Such a design 
standard may be expressed in symbols as B x C, where B Is 
the total number of objectives served by the particular Instruc- 
tional program, and C Is the total number of students to be 
trained or educated. The product BC Is the zero-reject Ideal: 
each student has successfully achieved all objectives. 

Let's take an example: assume we have a mathematics pro- 
gram whose total output is encompassed by 60 objectives, and 
we have 30 students to serve. If each of the students achieves 
all of the mathematics objectives, the zero-reject Ideal, In 
numerical terms, would be l800. 

It goes without saying that not everyone will achieve the 
zero-reject Idea?.. The control function, however will provide 
the actual achievement allowing us to make improvement decisions 
on the basis of these data. 

To pursue our mathematics example further, suppose that the 
30 St' dents actually achieve an aggregate of 1350 mathematics 
objec+lves (total number of objectives achieved times number of 
students). Let's call the actual number of objectives achieved 
ty students, A. Now, we have the Ingredients of the basic design 
formula for an effectiveness Index: 

EI -B^ 

where EI lb effectiveness of an Instructional system. In our 
example, 

EI = 2350 X 100 

or 75. (The 100 Is used In the formula to avoid the use of 
decimals.) The 75 EI says the class was 75 percent effective In 
achieving the zero-reject Ideal. Is this good or bad? It Is 
neither. What we have Is a baseline upon which to evaluate our 
efforts toward achieving an ideal. Whether 75 percent is good 
or bad can only be determined by considering many factors. 

Cost-Effectiveness Formula 

The dimensionality of the effectiveness formula can be 
increased by factoring in cost. Unfortunately, the present 
state of the art in cost accounting in education is not at 
a high level. Nevertheless, the development of cost-effec- 
tiveness ratios should receive high priority. The low state 
of the art, notwithstanding, it is possible to develop a 
highly useful cost -effectiveness formula, while allowing for 
the impreclseness of instructional program cost figures. 
The formula is as follows: 
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C/EI « EJL X 1000 EI » 75 C - $20,000 

c 

.where C is the computed cost for any instructional program in 
a given classroom. Using the effectiveness index from a 
hypothetical classroom, on the basis of an estimated cost of 
$20,000 to provide reading instruction, the figures would be 
computed as follows: 

C/EI ■ ' ''^iim" C/EI . 3.75 

Suppose figures on another classroom — with a comparable 
population — were as follows: 



EI « 68 C = $15,700 

Thus, 

C/EI = 68 X 1000 , c/EI = 4.33 
15,700 



The higher the C/EI, the greater the cost-effectiveness. 



The Spirit of the Audit 

The Germans have a word: Zeitgeist ^ which translates to 
"spirit of the age." It is imperative that educators and the 
public develop and promulgate the correct spirit of the educa- 
tional audit. The audit should not be punitive but facilita- 
tive. It might be useful to internalize an image of the audit 
as a flip card, having on one side, "audit". When the card is 
flipped over, it reads: " for the purpose of feedback and rede - 
sign " . For the purpose of feedback, so that people may note 
deviations from standards and possess sufficient information 
to redesign systems or programs. In its finest expression, 
the audit contributes to development of synergistic organiza- 
tions — organizations that place a high premium on collabora- 
tion and cooperation, for the purpose of achieving results. 

Gene Glass suggests that the evaluator might adopt the 
attitude of the therapist, who is inclined to temper Judgment 
with such statements as: "I may be wrong but . . .", or "How 
do you feel about the possibility that . . .?" Such an approach 
by evaluators would not only reduce threat on the part of 
teachers and principals, but would open up additional communi- 
cation and foster greater acceptance of necessary change. 

In the final analysis, the whole spirit of evaluation and 
audit may be captured in the statement: The purpose of evalua- 
tion is to improve , not to prove . 
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Educational Measurements and the Absurd 

The Jefferson Airplane had a song that went: 

Something's happening here. 
What It Is ain't exactly clear. 

There Is obviously a high degree of opacity regarding 
educational measurement. Much that happens would be funny If 
the consequences were not so serious. But then, the only way 
matters may be appreciated Is through humor, or within a context 
of the absurd. 

Consider the use of standardized norm-referenced achieve- 
ment tests. These tests are designed to measure Individual 
differences; not the effects of an Instructional program, except 
In gross terms. Take the classroom teacher who Is responsive 
to achievement discrepancies as measured by the standardized, 
normative tests. What does the conscientious teacher do? Naturally, 
teachers provide Instruction to overcome noted discrepancies. 
And If the teacher Is successful, what happens? The Item 
"taught to" Is Invalidated and thrown out by the test makers 
because if there are enough conscientious teachers around, the 
Item will no longer discriminate. Thus, as soon as teachers are 
successful, their success Is negated. Perversely, normative 
tests are directed at measuring failure at least for a prescribed 
number of the population. Children, under the normative system, 
are stretched on a rack of self-fulfilling prophecy known as a 
normal distribution. Net surprisingly, failure for some students 
Is pre-ordained. Another unfortunate consequence of the normal 
distribution syndrome ±s that teachers also become tragic 
victims of "fate control," a state of mind that surrenders per- 
sonal Initiative to outside forces beyond one's control. The 
antedote to this norm-Induced Weltanschauung of quiet desperation 
Is adoption of a zero-relect Ideal and learnln'- mastery, as 
espoused by B. S. Bloom. 5 

This is not to suggest we throw out normative tests; It Is a 
strong plea to put these tests Into proper perspective. For one 
thing, the public still has greac confidence In normative tests; 
although recent discoveries of functional Incompetence among 
students and graduates have eroded this confidence somewhat. 

It follows, assessments used In auditing should rely on a 
number of approaches: normative tests, criterion-referenced tests, 
teacher-created tests, and professional judgment. And In our 
pursuit of precision, care must be constantly exercised to 
utilize fully the latter. 



5/ Every Kid Can ; Learning for Mastery, Learning Support Systems 
(Washington, D.C. 20008, Suite 21, 21^1 Wyoming Ave., N.W. , 
^ 1973). 
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Specific Responsibilities of Quality Control and Quality 
Assurance Unit 

We have discussed the Inseparability, even the synergis- 
tic linkage, of quality control and quality assurance and 
pointed out that the content of any quallty-assurance program 
Is the quality-control processes In an organization. Therefore, 
major units In education should establish a quality-control 
function. The general responsibilities of the unit are: 

(1) Documenting the quality specifications 
Inherent In quality design; 

(2) Verifying conformance to design; 

(3) Reporting and triggering efforts to overcome 
sporadic probleir*^; 

(4) Working cooperatively In planning to master 
chronic problems. 

The specific responsibilities of the unit are: 

(1) Developing test situations; 

(2) Supervising the testing; 

(3) Processing and Interpreting test data; 

(4) Formally communicating the results in a 
report; 

(5) Providing assistance and training to 
personnel to meet the new rigors of the 
zero-reject mind-set. 



The Importance of the Pre-Audit 

In the pre-audit stage, the organization to be audited 
develops a clear set of specifications, procedures, and guide- 
lines for the audit, the substance of which constitutes the 
basic material for a contract with an external auditor. Mat- 
ters relating to instrumentation, data analysis and interpre- 
tation, and security of data should be spelled out. Every 
attempt should be made so that audit results are not used 
Improperly. The spirit of the audit shojld be articulated. 
It is also Important to Include a code of ethics for the audit. 



A Management Audit as a First Step 

Most school districts are not ready to conduct a full- 



id 
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scale educational program audit j the reason being chat most 
do not have, sound quality-control procedures. For those 01s- 
trlcts that find themselves In such a predicament, a :anage- 
ment audit Is suggested. In such an audit, an atteiipt Is 
made to look at overall management procedures as they relate 
to program design, logistics, and support. 



Summary 

Sound quality-control and quality-assurance practices 
represent a quantum Jump toward increased educational 
eff actlveness. Control is concerned with monitoring and re- 
sponding to conditions as they occur (formative); assurance is 
a summatlve process to provide feedback for redesign. The 
intentions of both processes must be viewed as positive end 
necessary* if our organizations are to "learn" and change for 
the better. Ideally, the net effect ox an educational audit 
will be an emergence of an increasing number of success 
strategies. Equally important, mistakes or discrepancies 
when they occur, are likely to be reduced in their magnitude. 
The audit, properly administered and reported, will go a long 
way toward strengthening che public confidence in our institu- 
tions . 

Before launching programs of educational auditing, 
qv^allty control, and qualit;/ assurance, the reader is well 
advl&ed to study the instructive words of Aaron V/lldavsky 
in his "Recipe for Revolution": 

Promise a lot; deliver a little. Teach 
people to believe that they will be much better 
off; but let there be no dramatic improvement. 
Try a variety of small programs but marginal in 
impact and severely unc^-^r financed . Avoid any 
attempted solution comparable in size to the g 
dimensions of the problem you are trying to solve... 

The challenge facing educators is greater than ever 
before, calling for extraordinary leadeishlp. Perhaps, what 
is most needed is a slight alter?>cion of our language; sub- 
stituting the word challenge for the word problem. Such a 
substitution constitutes not a transmogrification of language, 
but a transillence. With a seemingly simple (though difficult) 
alteration in viewpoint, we begin to see problems as sources 
of social energy which, to be maintained, must be rooted in 
optimism.. As John Piatt states: 

We see the gap between what is and what might be 
as the only source of power in the world. 



6/ Aaron Wildavsky, The Politics of the Budgetary Process 
(Boston: Little BrovinTTW^ 
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